build: bash shell for production docker builds #6469

eatyourgreens · 2024-11-14T22:43:09Z

This adds a bash shell for production docker builds, and renames the previous shell service to dev-shell.

It also optimises the Dockerfile for faster builds.

Please request review from @zooniverse/frontend team or an individual member of that team.

Linked Issue and/or Talk Post

build: docker instructions and fixes #6465.

How to Review

Similar to #6465, but with the changes here you should be able to get CLI shells into both production and dev installs of the monorepo:

docker compose build
docker compose run --rm dev-shell
docker compose run --rm prod-shell
docker compose down

I've been using these to run du -sh * inside a production build, and have a look at how big each directory is.

docker compose run --rm prod-shell
[+] Creating 1/0
 ✔ Network front-end-monorepo_default  Created                                                                                                      0.0s 
/usr/src # du -sh *
790.3M	node_modules
4.0K	package.json
641.7M	packages
824.0K	yarn.lock
/usr/src # du -sh packages/*
8.0K	packages/app-content-pages
327.3M	packages/app-project
302.3M	packages/app-root
48.0K	packages/lib-async-states
6.5M	packages/lib-classifier
660.0K	packages/lib-content
60.0K	packages/lib-grommet-theme
344.0K	packages/lib-panoptes-js
1.7M	packages/lib-react-components
236.0K	packages/lib-subject-viewers
2.6M	packages/lib-user
48.0K	packages/tools-standard
/usr/src # exit
docker compose down

When I run ls node_modules inside the production container, I don't see @storybook, mocha or webpack, so I think the production Yarn install is working, and only installing production dependencies. This might be useful for checking whether that's still true after moving to Yarn 4.

You should also be able to inspect the node_modules-dev and node_modules-prod volumes in Docker Desktop, to see the difference between production and development images.

Checklist

PR Creator - Please cater the checklist to fit the review needed for your code changes.
PR Reviewer - Use the checklist during your review. Each point should be checkmarked or discussed before PR approval.

General

Tests are passing locally and on Github
Documentation is up to date and changelog has been updated if appropriate
You can yarn panic && yarn bootstrap or docker-compose up --build and FEM works as expected
FEM works in all major desktop browsers: Firefox, Chrome, Edge, Safari (Use Browserstack account as needed)
FEM works in a mobile browser

coveralls · 2024-11-14T22:50:15Z

coverage: 77.851% (-0.02%) from 77.868%
when pulling 377b655 on eatyourgreens:docker-compose-builds
into c146bdf on zooniverse:master.

eatyourgreens · 2024-11-14T22:59:31Z

packages/app-root/README.md

@@ -27,7 +27,7 @@ docker compose up -d
 # stop the local services when you're finished
 docker compose down
 # run the tests
-docker compose run --rm project test
+docker compose run --rm root test


Small typo. The Next.js app service is called root in docker-compose.yml.

eatyourgreens · 2024-11-14T23:02:08Z

docker-compose.yml

+  prod-shell:
+    image: front-end-monorepo_prod:latest
+    build:
+      context: ./
+      args:
+        - NODE_ENV=production
+        - PANOPTES_ENV=production
+        - NEXT_TELEMETRY_DISABLED=1
+        - APP_ENV=development
+    command:
+      - "/bin/sh"


This runs bin/sh inside the front-end-monorepo_prod container, giving you a bash shell so that you can poke around the file system and debug the production build.

dev-shell does the same thing for front-end-monorepo_dev, which has dev dependencies installed (so you can also run tests from that shell.)

eatyourgreens · 2024-11-14T23:03:33Z

docker-compose.yml

    image: front-end-monorepo_dev:latest
-    volumes:
-      - node_modules:/usr/src/node_modules


~~I removed this to make it easier to debug the node package installs. Otherwise, you're just looking at the node_modules directory from your local machine.~~ I'm going to avoid changes to existing services here. Just add a new service for the production bash shell, and rename the existing service (and volume.)

eatyourgreens · 2024-11-14T23:06:51Z

Here's the output of docker images on this branch.

front-end-monorepo % docker images
REPOSITORY                                   TAG                   IMAGE ID       CREATED          SIZE
front-end-monorepo_prod                      latest                26d2798ccb75   33 minutes ago   1.43GB
front-end-monorepo_dev                       latest                0c20a73cd0f3   35 minutes ago   2.07GB

eatyourgreens · 2024-11-15T14:19:51Z

packages/app-project/docker-compose.yml

+      - node_modules:/usr/src/node_modules
+volumes:
+  node_modules:


Shared volumes can be convenient for local development. You can inspect them in the Docker Desktop app. This one should contain all the installed Node development packages.

I added two node_modules volumes, so you can compare production vs. development installs in Docker Desktop.

This adds a bash shell for production docker builds, and renames the previous `shell` service to `dev-shell`.

- Add the lockfile last, so that previous layers are cached. - Copy files from the builder to the production app runner.

eatyourgreens · 2024-11-19T09:38:07Z

Dockerfile

@@ -40,6 +38,8 @@ ADD lerna.json /usr/src/

 COPY ./packages /usr/src/packages

+ADD yarn.lock /usr/src/


yarn.lock changes quite frequently, so should be added to the image after all other files.

eatyourgreens · 2024-11-19T09:39:17Z

Dockerfile

+COPY --from=builder /usr/src/package.json /usr/src/package.json

-COPY .yarn /usr/src/.yarn
+COPY --from=builder /usr/src/.yarn /usr/src/.yarn

-ADD .yarnrc /usr/src/
+COPY --from=builder /usr/src/.yarnrc /usr/src/.yarnrc

 COPY --from=builder /usr/src/packages ./packages

+COPY --from=builder /usr/src/yarn.lock /usr/src/yarn.lock
+


Everything that the production runner uses should be copied from the builder image.

eatyourgreens mentioned this pull request Nov 14, 2024

Upgrade to Yarn Modern v4 #6446

Open

9 tasks

eatyourgreens force-pushed the docker-compose-builds branch from eef9b5c to cf7f63f Compare November 14, 2024 22:57

eatyourgreens commented Nov 14, 2024

View reviewed changes

eatyourgreens force-pushed the docker-compose-builds branch 3 times, most recently from 2640253 to 5321a95 Compare November 15, 2024 12:35

eatyourgreens commented Nov 15, 2024

View reviewed changes

eatyourgreens force-pushed the docker-compose-builds branch 2 times, most recently from 870783f to 7a72e98 Compare November 18, 2024 18:39

goplayoutside3 self-assigned this Nov 18, 2024

eatyourgreens added 3 commits November 18, 2024 21:09

build: bash shell for production docker builds

5ca2bdc

This adds a bash shell for production docker builds, and renames the previous `shell` service to `dev-shell`.

add the node_modules volume to the project app

01340c3

Separate node_modules volumes for production and development

0f24ff5

eatyourgreens force-pushed the docker-compose-builds branch from 7a72e98 to 0f24ff5 Compare November 18, 2024 21:09

Optimise the Docker build

377b655

- Add the lockfile last, so that previous layers are cached. - Copy files from the builder to the production app runner.

eatyourgreens commented Nov 19, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

build: bash shell for production docker builds #6469

build: bash shell for production docker builds #6469

eatyourgreens commented Nov 14, 2024 •

edited

Loading

coveralls commented Nov 14, 2024 •

edited

Loading

eatyourgreens Nov 14, 2024

eatyourgreens Nov 14, 2024

eatyourgreens Nov 14, 2024 •

edited

Loading

eatyourgreens commented Nov 14, 2024

eatyourgreens Nov 15, 2024 •

edited

Loading

eatyourgreens Nov 18, 2024

eatyourgreens Nov 19, 2024 •

edited

Loading

eatyourgreens Nov 19, 2024

		@@ -40,6 +38,8 @@ ADD lerna.json /usr/src/

		COPY ./packages /usr/src/packages

		ADD yarn.lock /usr/src/

build: bash shell for production docker builds #6469

Are you sure you want to change the base?

build: bash shell for production docker builds #6469

Conversation

eatyourgreens commented Nov 14, 2024 • edited Loading

Linked Issue and/or Talk Post

How to Review

Checklist

General

coveralls commented Nov 14, 2024 • edited Loading

eatyourgreens Nov 14, 2024

Choose a reason for hiding this comment

eatyourgreens Nov 14, 2024

Choose a reason for hiding this comment

eatyourgreens Nov 14, 2024 • edited Loading

Choose a reason for hiding this comment

eatyourgreens commented Nov 14, 2024

eatyourgreens Nov 15, 2024 • edited Loading

Choose a reason for hiding this comment

eatyourgreens Nov 18, 2024

Choose a reason for hiding this comment

eatyourgreens Nov 19, 2024 • edited Loading

Choose a reason for hiding this comment

eatyourgreens Nov 19, 2024

Choose a reason for hiding this comment

eatyourgreens commented Nov 14, 2024 •

edited

Loading

coveralls commented Nov 14, 2024 •

edited

Loading

eatyourgreens Nov 14, 2024 •

edited

Loading

eatyourgreens Nov 15, 2024 •

edited

Loading

eatyourgreens Nov 19, 2024 •

edited

Loading